Rating organization cybersecurity using active and passive external reconnaissance

ABSTRACT

A system for cybersecurity rating using active and passive external reconnaissance, that uses a web crawler that sends message prompts to external hosts and receives responses from external hosts, a time-series data store that produces time-series data from the message responses, and a directed computational graph module that analyzes the time-series data to produce a weighted score representing the overall cybersecurity state of an organization.

CROSS-REFERENCE TO RELATED APPLICATIONS

Application No. Date Filed Title Current Herewith RATING ORGANIZATIONapplication CYBERSECURITY USING ACTIVE AND PASSIVE EXTERNALRECONNAISSANCE Is a continuation of: 16/720,383 Dec. 19, 2019 RATINGORGANIZATION CYBERSECURITY USING ACTIVE AND PASSIVE EXTERNALRECONNAISSANCE Which is a continuation of: 15/823,363 Nov. 27, 2017RATING ORGANIZATION Patent Issue Date CYBERSECURITY USING ACTIVE AND10,560,483 Feb. 11, 2020 PASSIVE EXTERNAL RECONNAISSANCE Which is acontinuation-in-part of: 15/725,274 Oct. 4, 2017 APPLICATION OF ADVANCEDPatent Issue Date CYBERSECURITY THREAT MITIGATION 10,609,079 Mar. 31,2020 TO ROGUE DEVICES, PRIVILEGE ESCALATION, AND RISK-BASEDVULNERABILITY AND PATCH MANAGEMENT Which is a continuation-in-part of:15/655,113 Jul. 20, 2017 ADVANCED CYBERSECURITY THREAT Patent Issue DateMITIGATION USING BEHAVIORAL AND 10,735,456 Aug. 4, 2020 DEEP ANALYTICSWhich is a continuation-in-part of: 15/616,427 Jun. 7, 2017 RAPIDPREDICTIVE ANALYSIS OF VERY LARGE DATA SETS USING AN ACTOR- DRIVENDISTRIBUTED COMPUTATIONAL GRAPH Which is a continuation-in-part of:14/925,974 Oct. 28, 2015 RAPID PREDICTIVE ANALYSIS OF VERY LARGE DATASETS USING THE DISTRIBUTED COMPUTATIONAL GRAPH Current Herewith RATINGORGANIZATION application CYBERSECURITY USING ACTIVE AND PASSIVE EXTERNALRECONNAISSANCE Is a continuation of: 16/720,383 Dec. 19, 2019 RATINGORGANIZATION CYBERSECURITY USING ACTIVE AND PASSIVE EXTERNALRECONNAISSANCE Which is a continuation of: 15/823,363 Nov. 27, 2017RATING ORGANIZATION Patent Issue Date CYBERSECURITY USING ACTIVE AND10,560,483 Feb. 11, 2020 PASSIVE EXTERNAL RECONNAISSANCE Which is acontinuation-in-part of: 15/725,274 Oct. 4, 2017 APPLICATION OF ADVANCEDPatent Issue Date CYBERSECURITY THREAT MITIGATION 10,609,079 Mar. 31,2020 TO ROGUE DEVICES, PRIVILEGE ESCALATION, AND RISK-BASEDVULNERABILITY AND PATCH MANAGEMENT Which is a continuation-in-part of:15/655,113 Jul. 20, 2017 ADVANCED CYBERSECURITY THREAT Patent Issue DateMITIGATION USING BEHAVIORAL AND 10,735,456 Aug. 4, 2020 DEEP ANALYTICSWhich is a continuation-in-part of: 15/237,625 Aug. 15, 2016 DETECTIONMITIGATION AND Patent Issue Date: REMEDIATION OF CYBERATTACKS 10,248,910Apr. 2, 2019 EMPLOYING AN ADVANCED CYBER- DECISION PLATFORM Which is acontinuation-in-part of: 15/206,195 Jul. 8, 2016 ACCURATE AND DETAILEDMODELING OF SYSTEMS WITH LARGE COMPLEX DATASETS USING A DISTRIBUTEDSIMULATION ENGINE Which is a continuation-in-part of: 15/186,453 Jun.18, 2016 SYSTEM FOR AUTOMATED CAPTURE AND ANALYSIS OF BUSINESSINFORMATION FOR RELIABLE BUSINESS VENTURE OUTCOME PREDICTION Which is acontinuation-in-part of: 15/166,158 May 26, 2016 SYSTEM FOR AUTOMATEDCAPTURE AND ANALYSIS OF BUSINESS INFORMATION FOR SECURITY ANDCLIENT-FACING INFRASTRUCTURE RELIABILITY Which is a continuation-in-partof: 15/141,752 Apr. 28, 2016 SYSTEM FOR FULLY INTEGRATED Patent IssueDate CAPTURE, AND ANALYSIS OF BUSINESS 10,860,962 Dec. 8, 2020INFORMATION RESULTING IN PREDICTIVE DECISION MAKING AND SIMULATION Whichis a continuation-in-part of: 15/091,563 Apr. 5, 2016 SYSTEM FORCAPTURE, ANALYSIS AND Patent Issue Date: STORAGE OF TIME SERIES DATAFROM 10,204,147 Feb. 12, 2019 SENSORS WITH HETEROGENEOUS REPORT INTERVALPROFILES And is a continuation-in-part of: 14/986,536 Dec. 31, 2015DISTRIBUTED SYSTEM FOR LARGE Patent Issue Date: VOLUME DEEP WEB DATAEXTRACTION 10,210,255 Feb. 19, 2019 And is a continuation-in-part of:14/925,974 Oct. 28, 2015 RAPID PREDICTIVE ANALYSIS OF VERY LARGE DATASETS USING THE DISTRIBUTED COMPUTATIONAL GRAPH the entire specificationof each of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The disclosure relates to the field of computer management, and moreparticularly to the field of cybersecurity and threat analytics.

Discussion of the State of the Art

IPv4 address space consists of 2³², or almost 4.3 billion possible IPaddresses, which is within the giga-order of magnitude. Today'scomputers have CPUs with gigahertz clock rates and gigabytes of RAM andstorage. Today's networks allow band-widths exceeding 1 gigabit persecond. This makes iterations over the entire IPv4 space possible withina comparatively short time period. Today's port scanning software isable to perform massive port scans up to the complete IPv4 space withinminutes. Once a device is connected to the Internet, it will be almostimmediately scanned for open ports and services. Most of these scans aredone by anonymous individuals, in particular targeting at finding andexploiting system vulnerabilities. However, there is also a variety ofindividuals and organizations openly practicing massive port scanningand pursuing different objectives.

What is needed is a system and method for performing active and passiveexternal reconnaissance using network scanning, that feeds informationinto a distributed computational graph data pipeline to produce hybridgraph/time-series data for use in producing cybersecurity ratings fororganizations.

SUMMARY OF THE INVENTION

Accordingly, the inventor has developed a system and method for ratingorganization cybersecurity using active and passive externalreconnaissance.

The aspects described herein provide a system and methods forcybersecurity rating using active and passive external reconnaissance,comprising a web crawler that send message prompts to external hosts andreceives responses from external hosts, a time-series data store thatproduces time-series data from the message responses, and a directedcomputational graph module that analyzes the time-series data to producea weighted score representing the overall cybersecurity state of anorganization.

According to one aspect, an advanced cyber decision platform forexternal network reconnaissance and cybersecurity rating, the platformcomprising: a time-series data store comprising at least a processor, amemory, and a plurality of programming instructions stored in the memoryand operating on the processor, wherein the programmable instructions,when operating on the processor, cause the processor to: receive aplurality of response messages from external hosts connected via anetwork; produce time-series data based on at least a portion of areceived response message; a directed computational graph modulecomprising at least a processor, a memory, and a plurality ofprogramming instructions stored in the memory and operating on theprocessor, wherein the programmable instructions, when operating on theprocessor, cause the processor to: perform a plurality of analysis andtransformation operations on at least a portion of the time-series data;produce a weighted score based at least in part on the output of atleast a portion of the analysis and transformation operations; a webcrawler comprising at least a processor, a memory, and a plurality ofprogramming instructions stored in the memory and operating on theprocessor, wherein the programmable instructions, when operating on theprocessor, cause the processor to: transmit a message prompt to anexternal host via a network; receive a message response from theexternal host; and provide at least a portion of the message response tothe time-series data store, is disclosed.

According to another aspect, a method for external networkreconnaissance and cybersecurity rating, comprising the steps of: a)transmitting, using a web crawler, a plurality of message prompts to aplurality of external hosts via a network; b) receiving at least amessage response from an external host; c) producing, using atime-series data store, time-series data based at least in part on atleast a portion of the received message response; d) performing, using adirected computational graph module, a plurality of analysis andtransformation operations on at least a portion of the time-series data;and e) producing a weighted score based at least in part on the outputof at least a portion of the analysis and transformation operations, isdisclosed.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawings illustrate several aspects and, together withthe description, serve to explain the principles of the inventionaccording to the aspects. It will be appreciated by one skilled in theart that the particular arrangements illustrated in the drawings aremerely exemplary, and are not to be considered as limiting of the scopeof the invention or the claims herein in any way.

FIG. 1 is a block diagram of an exemplary system architecture for anadvanced cyber decision platform for external network reconnaissance andcybersecurity rating.

FIG. 2A is a block diagram showing general steps for performing passivenetwork reconnaissance.

FIG. 2B is a process diagram showing a general flow of a process forperforming active reconnaissance using DNS leak information collection.

FIG. 2C is a process diagram showing a general flow of a process forperforming active reconnaissance using web application and technologyreconnaissance.

FIG. 2D is a process diagram showing a general flow of a process forproducing a cybersecurity rating using reconnaissance data.

FIG. 3 is a process diagram showing business operating system functionsin use to mitigate cyberattacks.

FIG. 4 is a process flow diagram of a method for segmenting cyberattackinformation to appropriate corporation parties.

FIG. 5 is a diagram of an exemplary architecture for a system for rapidpredictive analysis of very large data sets using an actor-drivendistributed computational graph, according to one aspect.

FIG. 6 is a diagram of an exemplary architecture for a system for rapidpredictive analysis of very large data sets using an actor-drivendistributed computational graph, according to one aspect.

FIG. 7 is a diagram of an exemplary architecture for a system for rapidpredictive analysis of very large data sets using an actor-drivendistributed computational graph, according to one aspect.

FIG. 8 is a flow diagram of an exemplary method for cybersecuritybehavioral analytics, according to one aspect.

FIG. 9 is a flow diagram of an exemplary method for measuring theeffects of cybersecurity attacks, according to one aspect.

FIG. 10 is a flow diagram of an exemplary method for continuouscybersecurity monitoring and exploration, according to one aspect.

FIG. 11 is a flow diagram of an exemplary method for mapping acyber-physical system graph, according to one aspect.

FIG. 12 is a flow diagram of an exemplary method for continuous networkresilience scoring, according to one aspect.

FIG. 13 is a flow diagram of an exemplary method for cybersecurityprivilege oversight, according to one aspect.

FIG. 14 is a flow diagram of an exemplary method for cybersecurity riskmanagement, according to one aspect.

FIG. 15 is a flow diagram of an exemplary method for mitigatingcompromised credential threats, according to one aspect.

FIG. 16 is a flow diagram of an exemplary method for dynamic network androgue device discovery, according to one aspect.

FIG. 17 is a flow diagram of an exemplary method for Kerberos “goldenticket” attack detection, according to one aspect.

FIG. 18 is a flow diagram of an exemplary method for risk-basedvulnerability and patch management, according to one aspect.

FIG. 19 is a block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 20 is a block diagram illustrating an exemplary logicalarchitecture for a client device.

FIG. 21 is a block diagram illustrating an exemplary architecturalarrangement of clients, servers, and external services.

FIG. 22 is another block diagram illustrating an exemplary hardwarearchitecture of a computing device.

DETAILED DESCRIPTION

The inventor has conceived, and reduced to practice, a system and methodfor rating organization cybersecurity using active and passive externalreconnaissance.

One or more different aspects may be described in the presentapplication. Further, for one or more of the aspects described herein,numerous alternative arrangements may be described; it should beappreciated that these are presented for illustrative purposes only andare not limiting of the aspects contained herein or the claims presentedherein in any way. One or more of the arrangements may be widelyapplicable to numerous aspects, as may be readily apparent from thedisclosure. In general, arrangements are described in sufficient detailto enable those skilled in the art to practice one or more of theaspects, and it should be appreciated that other arrangements may beutilized and that structural, logical, software, electrical and otherchanges may be made without departing from the scope of the particularaspects. Particular features of one or more of the aspects describedherein may be described with reference to one or more particular aspectsor figures that form a part of the present disclosure, and in which areshown, by way of illustration, specific arrangements of one or more ofthe aspects. It should be appreciated, however, that such features arenot limited to usage in the one or more particular aspects or figureswith reference to which they are described. The present disclosure isneither a literal description of all arrangements of one or more of theaspects nor a listing of features of one or more of the aspects thatmust be present in all arrangements.

Headings of sections provided in this patent application and the titleof this patent application are for convenience only, and are not to betaken as limiting the disclosure in any way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or morecommunication means or intermediaries, logical or physical.

A description of an aspect with several components in communication witheach other does not imply that all such components are required. To thecontrary, a variety of optional components may be described toillustrate a wide variety of possible aspects and in order to more fullyillustrate one or more aspects. Similarly, although process steps,method steps, algorithms or the like may be described in a sequentialorder, such processes, methods and algorithms may generally beconfigured to work in alternate orders, unless specifically stated tothe contrary. In other words, any sequence or order of steps that may bedescribed in this patent application does not, in and of itself,indicate a requirement that the steps be performed in that order. Thesteps of described processes may be performed in any order practical.Further, some steps may be performed simultaneously despite beingdescribed or implied as occurring non-simultaneously (e.g., because onestep is described after the other step). Moreover, the illustration of aprocess by its depiction in a drawing does not imply that theillustrated process is exclusive of other variations and modificationsthereto, does not imply that the illustrated process or any of its stepsare necessary to one or more of the aspects, and does not imply that theillustrated process is preferred. Also, steps are generally describedonce per aspect, but this does not mean they must occur once, or thatthey may only occur once each time a process, method, or algorithm iscarried out or executed. Some steps may be omitted in some aspects orsome occurrences, or some steps may be executed more than once in agiven aspect or occurrence.

When a single device or article is described herein, it will be readilyapparent that more than one device or article may be used in place of asingle device or article. Similarly, where more than one device orarticle is described herein, it will be readily apparent that a singledevice or article may be used in place of the more than one device orarticle.

The functionality or the features of a device may be alternativelyembodied by one or more other devices that are not explicitly describedas having such functionality or features. Thus, other aspects need notinclude the device itself.

Techniques and mechanisms described or referenced herein will sometimesbe described in singular form for clarity. However, it should beappreciated that particular aspects may include multiple iterations of atechnique or multiple instantiations of a mechanism unless notedotherwise. Process descriptions or blocks in figures should beunderstood as representing modules, segments, or portions of code whichinclude one or more executable instructions for implementing specificlogical functions or steps in the process. Alternate implementations areincluded within the scope of various aspects in which, for example,functions may be executed out of order from that shown or discussed,including substantially concurrently or in reverse order, depending onthe functionality involved, as would be understood by those havingordinary skill in the art.

Definitions

As used herein, “graph” is a representation of information andrelationships, where each primary unit of information makes up a “node”or “vertex” of the graph and the relationship between two nodes makes upan edge of the graph. Nodes can be further qualified by the connectionof one or more descriptors or “properties” to that node. For example,given the node “James R,” name information for a person, qualifyingproperties might be “183 cm tall”, “DOB Aug. 13, 1965” and “speaksEnglish”. Similar to the use of properties to further describe theinformation in a node, a relationship between two nodes that forms anedge can be qualified using a “label”. Thus, given a second node “ThomasG,” an edge between “James R” and “Thomas G” that indicates that the twopeople know each other might be labeled “knows.” When graph theorynotation (Graph=(Vertices, Edges)) is applied this situation, the set ofnodes are used as one parameter of the ordered pair, V and the set of 2element edge endpoints are used as the second parameter of the orderedpair, E. When the order of the edge endpoints within the pairs of E isnot significant, for example, the edge James R, Thomas G is equivalentto Thomas G, James R, the graph is designated as “undirected.” Undercircumstances when a relationship flows from one node to another in onedirection, for example James R is “taller” than Thomas G, the order ofthe endpoints is significant. Graphs with such edges are designated as“directed.” In the distributed computational graph system,transformations within transformation pipeline are represented asdirected graph with each transformation comprising a node and the outputmessages between transformations comprising edges. Distributedcomputational graph stipulates the potential use of non-lineartransformation pipelines which are programmatically linearized. Suchlinearization can result in exponential growth of resource consumption.The most sensible approach to overcome possibility is to introduce newtransformation pipelines just as they are needed, creating only thosethat are ready to compute. Such method results in transformation graphswhich are highly variable in size and node, edge composition as thesystem processes data streams. Those familiar with the art will realizethat transformation graph may assume many shapes and sizes with a vasttopography of edge relationships. The examples given were chosen forillustrative purposes only and represent a small number of the simplestof possibilities. These examples should not be taken to define thepossible graphs expected as part of operation of the invention

As used herein, “transformation” is a function performed on zero or morestreams of input data which results in a single stream of output whichmay or may not then be used as input for another transformation.Transformations may comprise any combination of machine, human ormachine-human interactions Transformations need not change data thatenters them, one example of this type of transformation would be astorage transformation which would receive input and then act as a queuefor that data for subsequent transformations. As implied above, aspecific transformation may generate output data in the absence of inputdata. A time stamp serves as a example. In the invention,transformations are placed into pipelines such that the output of onetransformation may serve as an input for another. These pipelines canconsist of two or more transformations with the number oftransformations limited only by the resources of the system.Historically, transformation pipelines have been linear with eachtransformation in the pipeline receiving input from one antecedent andproviding output to one subsequent with no branching or iteration. Otherpipeline configurations are possible. The invention is designed topermit several of these configurations including, but not limited to:linear, afferent branch, efferent branch and cyclical.

A “database” or “data storage subsystem” (these terms may be consideredsubstantially synonymous), as used herein, is a system adapted for thelong-term storage, indexing, and retrieval of data, the retrievaltypically being via some sort of querying interface or language.“Database” may be used to refer to relational database managementsystems known in the art, but should not be considered to be limited tosuch systems. Many alternative database or data storage systemtechnologies have been, and indeed are being, introduced in the art,including but not limited to distributed non-relational data storagesystems such as Hadoop, column-oriented databases, in-memory databases,and the like. While various aspects may preferentially employ one oranother of the various data storage subsystems available in the art (oravailable in the future), the invention should not be construed to be solimited, as any data storage architecture may be used according to theaspects. Similarly, while in some cases one or more particular datastorage needs are described as being satisfied by separate components(for example, an expanded private capital markets database and aconfiguration database), these descriptions refer to functional uses ofdata storage systems and do not refer to their physical architecture.For instance, any group of data storage systems of databases referred toherein may be included together in a single database management systemoperating on a single machine, or they may be included in a singledatabase management system operating on a cluster of machines as isknown in the art. Similarly, any single database (such as an expandedprivate capital markets database) may be implemented on a singlemachine, on a set of machines using clustering technology, on severalmachines connected by one or more messaging systems known in the art, orin a master/slave arrangement common in the art. These examples shouldmake clear that no particular architectural approaches to databasemanagement is preferred according to the invention, and choice of datastorage technology is at the discretion of each implementer, withoutdeparting from the scope of the invention as claimed.

A “data context”, as used herein, refers to a set of argumentsidentifying the location of data. This could be a Rabbit queue, a .csvfile in cloud-based storage, or any other such location reference excepta single event or record. Activities may pass either events or datacontexts to each other for processing. The nature of a pipeline allowsfor direct information passing between activities, and data locations orfiles do not need to be predetermined at pipeline start.

A “pipeline”, as used herein and interchangeably referred to as a “datapipeline” or a “processing pipeline”, refers to a set of data streamingactivities and batch activities. Streaming and batch activities can beconnected indiscriminately within a pipeline. Events will flow throughthe streaming activity actors in a reactive way. At the junction of astreaming activity to batch activity, there will exist aStreamBatchProtocol data object. This object is responsible fordetermining when and if the batch process is run. One or more of threepossibilities can be used for processing triggers: regular timinginterval, every N events, or optionally an external trigger. The eventsare held in a queue or similar until processing. Each batch activity maycontain a “source” data context (this may be a streaming context if theupstream activities are streaming), and a “destination” data context(which is passed to the next activity). Streaming activities may have anoptional “destination” streaming data context (optional meaning:caching/persistence of events vs. ephemeral), though this should not bepart of the initial implementation.

Conceptual Architecture

FIG. 1 is a block diagram of an advanced cyber decision platform forexternal network reconnaissance and cybersecurity rating. Client accessto the system 105 for specific data entry, system control and forinteraction with system output such as automated predictive decisionmaking and planning and alternate pathway simulations, occurs throughthe system's distributed, extensible high bandwidth cloud interface 110which uses a versatile, robust web application driven interface for bothinput and display of client-facing information via network 107 andoperates a data store 112 such as, but not limited to MONGODB™,COUCHDB™, CASSANDRA™ or REDIS™ according to various arrangements. Muchof the business data analyzed by the system both from sources within theconfines of the client business, and from cloud based sources, alsoenter the system through the cloud interface 110, data being passed tothe connector module 135 which may possess the API routines 135 a neededto accept and convert the external data and then pass the normalizedinformation to other analysis and transformation components of thesystem, the directed computational graph module 155, high volume webcrawler module 115, multidimensional time series database (MDTSDB) 120and the graph stack service 145. The directed computational graph module155 retrieves one or more streams of data from a plurality of sources,which includes, but is in no way not limited to, a plurality of physicalsensors, network service providers, web based questionnaires andsurveys, monitoring of electronic infrastructure, crowd sourcingcampaigns, and human input device information. Within the directedcomputational graph module 155, data may be split into two identicalstreams in a specialized pre-programmed data pipeline 155 a, wherein onesub-stream may be sent for batch processing and storage while the othersub-stream may be reformatted for transformation pipeline analysis. Thedata is then transferred to the general transformer service module 160for linear data transformation as part of analysis or the decomposabletransformer service module 150 for branching or iterativetransformations that are part of analysis. The directed computationalgraph module 155 represents all data as directed graphs where thetransformations are nodes and the result messages betweentransformations edges of the graph. The high volume web crawling module115 uses multiple server hosted preprogrammed web spiders, which whileautonomously configured are deployed within a web scraping framework 115a of which SCRAPY™ is an example, to identify and retrieve data ofinterest from web based sources that are not well tagged by conventionalweb crawling technology. The multiple dimension time series data storemodule 120 may receive streaming data from a large plurality of sensorsthat may be of several different types. The multiple dimension timeseries data store module may also store any time series data encounteredby the system such as but not limited to enterprise network usage data,component and system logs, performance data, network service informationcaptures such as, but not limited to news and financial feeds, and salesand service related customer data. The module is designed to accommodateirregular and high volume surges by dynamically allotting networkbandwidth and server processing channels to process the incoming data.Inclusion of programming wrappers 120 a for languages examples of whichare, but not limited to C++, PERL, PYTHON, and ERLANG™ allowssophisticated programming logic to be added to the default function ofthe multidimensional time series database 120 without intimate knowledgeof the core programming, greatly extending breadth of function. Dataretrieved by the multidimensional time series database (MDTSDB) 120 andthe high volume web crawling module 115 may be further analyzed andtransformed into task optimized results by the directed computationalgraph 155 and associated general transformer service 150 anddecomposable transformer service 160 modules. Alternately, data from themultidimensional time series database and high volume web crawlingmodules may be sent, often with scripted cuing information determiningimportant vertexes 145 a, to the graph stack service module 145 which,employing standardized protocols for converting streams of informationinto graph representations of that data, for example, open graphinternet technology although the invention is not reliant on any onestandard. Through the steps, the graph stack service module 145represents data in graphical form influenced by any pre-determinedscripted modifications 145 a and stores it in a graph-based data store145 b such as GIRAPH™ or a key value pair type data store REDIS™, orRIAK™, among others, all of which are suitable for storing graph-basedinformation.

Results of the transformative analysis process may then be combined withfurther client directives, additional business rules and practicesrelevant to the analysis and situational information external to thealready available data in the automated planning service module 130which also runs powerful information theory 130 a based predictivestatistics functions and machine learning algorithms to allow futuretrends and outcomes to be rapidly forecast based upon the current systemderived results and choosing each a plurality of possible businessdecisions. The using all available data, the automated planning servicemodule 130 may propose business decisions most likely to result is themost favorable business outcome with a usably high level of certainty.Closely related to the automated planning service module in the use ofsystem derived results in conjunction with possible externally suppliedadditional information in the assistance of end user business decisionmaking, the action outcome simulation module 125 with its discrete eventsimulator programming module 125 a coupled with the end user facingobservation and state estimation service 140 which is highly scriptable140 b as circumstances require and has a game engine 140 a to morerealistically stage possible outcomes of business decisions underconsideration, allows business decision makers to investigate theprobable outcomes of choosing one pending course of action over anotherbased upon analysis of the current available data.

When performing external reconnaissance via a network 107, web crawler115 may be used to perform a variety of port and service scanningoperations on a plurality of hosts. This may be used to targetindividual network hosts (for example, to examine a specific server orclient device) or to broadly scan any number of hosts (such as all hostswithin a particular domain, or any number of hosts up to the completeIPv4 address space). Port scanning is primarily used for gatheringinformation about hosts and services connected to a network, using probemessages sent to hosts that prompt a response from that host. Portscanning is generally centered around the transmission control protocol(TCP), and using the information provided in a prompted response a portscan can provide information about network and application layers on thetargeted host.

Port scan results can yield information on open, closed, or undeterminedports on a target host. An open port indicated that an application orservice is accepting connections on this port (such as ports used forreceiving customer web traffic on a web server), and these portsgenerally disclose the greatest quantity of useful information about thehost. A closed port indicates that no application or service islistening for connections on that port, and still provides informationabout the host such as revealing the operating system of the host, whichmay discovered by fingerprinting the TCP/IP stack in a response.Different operating systems exhibit identifiable behaviors whenpopulating TCP fields, and collecting multiple responses and matchingthe fields against a database of known fingerprints makes it possible todetermine the OS of the host even when no ports are open. Anundetermined port is one that does not produce a requested response,generally because the port is being filtered by a firewall on the hostor between the host and the network (for example, a corporate firewallbehind which all internal servers operate).

Scanning may be defined by scope to limit the scan according to twodimensions, hosts and ports. A horizontal scan checks the same port onmultiple hosts, often used by attackers to check for an open port on anyavailable hosts to select a target for an attack that exploits avulnerability using that port. This type of scan is also useful forsecurity audits, to ensure that vulnerabilities are not exposed on anyof the target hosts. A vertical scan defines multiple ports to examineon a single host, for example a “vanilla scan” which targets every portof a single host, or a “strobe scan” that targets a small subset ofports on the host. This type of scan is usually performed forvulnerability detection on single systems, and due to the single-hostnature is impractical for large network scans. A block scan combineselements of both horizontal and vertical scanning, to scan multipleports on multiple hosts. This type of scan is useful for a variety ofservice discovery and data collection tasks, as it allows a broad scanof many hosts (up to the entire Internet, using the complete IPv4address space) for a number of desired ports in a single sweep.

Large port scans involve quantitative research, and as such may betreated as experimental scientific measurement and are subject tomeasurement and quality standards to ensure the usefulness of results.To avoid observational errors during measurement, results must beprecise (describing a degree of relative proximity between individualmeasured values), accurate (describing relative proximity of measuredvalues to a reference value), preserve any metadata that accompanies themeasured data, avoid misinterpretation of data due to faulty measurementexecution, and must be well-calibrated to efficiently expose and addressissues of inaccuracy or misinterpretation. In addition to these basicrequirements, large volumes of data may lead to unexpected behavior ofanalysis tools, and extracting a subset to perform initial analysis mayhelp to provide an initial overview before working with the completedata set. Analysis should also be reproducible, as with all experimentalscience, and should incorporate publicly-available data to add value tothe comprehensibility of the research as well as contributing to a“common framework” that may be used to confirm results.

When performing a port scan, web crawler 115 may employ a variety ofsoftware suitable for the task, such as Nmap, ZMap, or masscan. Nmap issuitable for large scans as well as scanning individual hosts, andexcels in offering a variety of diverse scanning techniques. ZMap is anewer application and unlike Nmap (which is more general-purpose), ZMapis designed specifically with Internet-wide scans as the intent. As aresult, ZMap is far less customizable and relies on horizontal portscans for functionality, achieving fast scan times using techniques ofprobe randomization (randomizing the order in which probes are sent tohosts, minimizing network saturation) and asynchronous design (utilizingstateless operation to send and receive packets in separate processingthreads). Masscan uses the same asynchronous operation model of ZMap, aswell as probe randomization. In masscan however, a certain degree ofstatistical randomness is sacrificed to improve computation time forlarge scans (such as when scanning the entire IPv4 address space), usingthe BlackRock algorithm. This is a modified implementation of symmetricencryption algorithm DES, with fewer rounds and modulo operations inplace of binary ones to allow for arbitrary ranges and achieve fastercomputation time for large data sets.

Received scan responses may be collected and processed through aplurality of data pipelines 155 a to analyze the collected information.MDTSDB 120 and graph stack 145 may be used to produce a hybridgraph/time-series database using the analyzed data, forming a graph ofInternet-accessible organization resources and their evolving stateinformation over time. Customer-specific profiling and scanninginformation may be linked to CPG graphs (as described below in detail,referring to FIG. 11) for a particular customer, but this informationmay be further linked to the base-level graph of internet-accessibleresources and information. Depending on customer authorizations andlegal or regulatory restrictions and authorizations, techniques used mayinvolve both passive, semi-passive and active scanning andreconnaissance.

FIG. 2A is a block diagram showing general steps 200 for performingpassive network reconnaissance. It should be appreciated that the stepsillustrated and described may be performed in any order, and that stepsmay be added or omitted as needed for any particular reconnaissanceoperation. In a step 201, network address ranges and domains orsub-domains associated with a plurality of targets may be identified,for example to collect information for defining the scope of furtherscanning operations. In another step 202, external sites may beidentified to understand relationships between targets and otherthird-party content providers, such as trust relationships orauthoritative domain name service (DNS) resolution records. In anotherstep 203, individual people or groups may be identified using names,email addresses, phone numbers, or other identifying information thatmay be useful for a variety of social engineering activities. In anotherstep 204, technologies used may be identified, such as types or versionsof hardware or software used by an organization, and this may includecollecting and extracting information from job descriptions (forexample) to identify technologies in use by an organization (forexample, a job description for an administrator familiar with specificdatabase software indicates that said software is in use within theorganization). In another step 205, content of interest may beidentified, for example including web and email portals, log files,backup or archive files, and other forms of sensitive information thatmay be contained within HTML comments or client-side scripts, as may beuseful for vulnerability discovery and penetration testing activities.In another step 206, publicly-available information may be used toidentify vulnerabilities that may be exploited with further activepenetration testing.

FIG. 2B is a process diagram showing a general flow of a process 210 forperforming active reconnaissance using DNS leak information collection.In an initial step 211, publicly-available DNS leak disclosureinformation may be collected to maintain current information regardingknown leaks and vulnerabilities. In a next step 212, third-level domain(TLDR) information may be collected and used to report domain riskfactors, such as domains that do not resolve properly (due to malformedDNS records, for example). In a next step 213, a DNS trust map may becreated using a hybrid graph/time-series data structure, using a graphstack service 145 and MDTSDB 120. This trust map may be produced as theoutput of an extraction process performed by a DCG 155 through aplurality of data pipelines 155 a, analyzing collected data and mappingdata points to produce hybrid structured output representing each datapoint over time. In a final step 214, the trust map may then be analyzedto identify anomalies, for example using community detection algorithmsthat may discover when new references are being created, and this may beused to identify vulnerabilities that may arise as a byproduct of thereferential nature of a DNS hierarchy. In this manner, DCG pipelineprocessing and time-series data graphing may be used to identifyvulnerabilities that would otherwise be obscured within a large dataset.

FIG. 2C is a process diagram showing a general flow of a process 220 forperforming active reconnaissance using web application and technologyreconnaissance. In an initial step 221, a plurality of manual HTTPrequests may be transmitted to a host, for example to determine if a webserver is announcing itself, or to obtain an application version numberfrom an HTTP response message. In a next step 222, a robots.txt, used toidentify and communicate with web crawlers and other automated “bots”,may be searched for to identify portions of an application or site thatrobots are requested to ignore. In a next step 223, the host applicationlayer may be fingerprinted, for example using file extensions andresponse message fields to identify characteristic patterns or markersthat may be used to identify host or application details. In a next step224, publicly-exposed/admin pages may be checked, to determine if anyadministrative portals are exposed and therefore potentially-vulnerable,as well as to potentially determine administration policies orcapabilities based on exposed information. In a final step 225, anapplication may be profiled according to a particular toolset in use,such as WORDPRESS™ (for example) or other specific tools or plugins.

FIG. 2D is a process diagram showing a general flow of a process 230 forproducing a cybersecurity rating using reconnaissance data. In aninitial step 231, external reconnaissance may be performed using DNS andIP information as described above (referring to FIG. 2B), collectinginformation from DNS records, leak announcements, and publicly-availablerecords to produce a DNS trust map from collected information and theDCG-driven analysis thereof. In a next step 232, web and applicationrecon may be performed (as described in FIG. 2C), collecting informationon applications, sites, and publicly-available records. In a next step233, collected information over time may be analyzed for softwareversion numbers, revealing the patching frequency of target hosts andtheir respective applications and services. Using a hybrid time-seriesgraph, timestamps may be associated with ongoing changes to reveal theseupdates over time. In a next step 234, a plurality of additionalendpoints may be scanned, such as (for example, including but notlimited to) internet-of-things (IoT) devices that may be scanned andfingerprinted, end-user devices such as personal smartphones, tablets,or computers, or social network endpoints such as scraping content fromuser social media pages or feeds. User devices may be fingerprinted andanalyzed similar to organization hosts, and social media content may beretrieved such as collecting sentiment from services like TWITTER™ orLINKEDIN™, or analyzing job description listings and otherpublicly-available information. In a next step 235, open-sourceintelligence feeds may be checked, such as company IP addressblacklists, search domains, or information leaks (for example, posted topublic records such as PASTEBIN™). In a final step 236, collectedinformation from all sources may be scored according to a weightedsystem, producing an overall cybersecurity rating score based on theinformation collected and the analysis of that information to revealadditional insights, relationships, and vulnerabilities.

For example, in an exemplary scoring system similar to a credit rating,information from initial Internet recon operations may be assigned ascore up to 400 points, along with up to 200 additional points forweb/application recon results, 100 points for patch frequency, and 50points each for additional endpoints and open-source intel results. Thisyields a weighted score incorporating all available information from allscanned sources, allowing a meaningful and readily-appreciablerepresentation of an organization's overall cybersecurity strength.Additionally, as scanning may be performed repeatedly and resultscollected into a time-series hybrid data structure, this cybersecurityrating may evolve over time to continuously reflect the current state ofthe organization, reflecting any recent changes, newly-discovered orannounced vulnerabilities, software or hardware updates, newly-added orremoved devices or services, and any other changes that may occur.

FIG. 3 is a process diagram showing a general flow 300 of businessoperating system functions in use to mitigate cyberattacks. Inputnetwork data which may include network flow patterns 321, the origin anddestination of each piece of measurable network traffic 322, system logsfrom servers and workstations on the network 323, endpoint data 323 a,any security event log data from servers or available securityinformation and event (SIEM) systems 324, external threat intelligencefeeds 324 a, identity or assessment context 325, external network healthor cybersecurity feeds 326, Kerberos domain controller or ACTIVEDIRECTORY™ server logs or instrumentation 327 and business unitperformance related data 328, among many other possible data types forwhich the invention was designed to analyze and integrate, may pass into315 the business operating system 310 for analysis as part of its cybersecurity function. These multiple types of data from a plurality ofsources may be transformed for analysis 311, 312 using at least one ofthe specialized cybersecurity, risk assessment or common functions ofthe business operating system in the role of cybersecurity system, suchas, but not limited to network and system user privilege oversight 331,network and system user behavior analytics 332, attacker and defenderaction timeline 333, SIEM integration and analysis 334, dynamicbenchmarking 335, and incident identification and resolution performanceanalytics 336 among other possible cybersecurity functions; value atrisk (VAR) modeling and simulation 341, anticipatory vs. reactive costestimations of different types of data breaches to establish priorities342, work factor analysis 343 and cyber event discovery rate 344 as partof the system's risk analytics capabilities; and the ability to formatand deliver customized reports and dashboards 351, perform generalized,ad hoc data analytics on demand 352, continuously monitor, process andexplore incoming data for subtle changes or diffuse informationalthreads 353 and generate cyber-physical systems graphing 354 as part ofthe business operating system's common capabilities. Output 317 can beused to configure network gateway security appliances 361, to assist inpreventing network intrusion through predictive change to infrastructurerecommendations 362, to alert an enterprise of ongoing cyberattack earlyin the attack cycle, possibly thwarting it but at least mitigating thedamage 362, to record compliance to standardized guidelines or SLArequirements 363, to continuously probe existing network infrastructureand issue alerts to any changes which may make a breach more likely 364,suggest solutions to any domain controller ticketing weaknesses detected365, detect presence of malware 366, and perform one time or continuousvulnerability scanning depending on client directives 367. Theseexamples are, of course, only a subset of the possible uses of thesystem, they are exemplary in nature and do not reflect any boundariesin the capabilities of the invention.

FIG. 4 is a process flow diagram of a method for segmenting cyberattackinformation to appropriate corporation parties 400. As previouslydisclosed 200, 351, one of the strengths of the advanced cyber-decisionplatform is the ability to finely customize reports and dashboards tospecific audiences, concurrently is appropriate. This customization ispossible due to the devotion of a portion of the business operatingsystem's programming specifically to outcome presentation by moduleswhich include the observation and state estimation service 140 with itsgame engine 140 a and script interpreter 140 b. In the setting ofcybersecurity, issuance of specialized alerts, updates and reports maysignificantly assist in getting the correct mitigating actions done inthe most timely fashion while keeping all participants informed atpredesignated, appropriate granularity. Upon the detection of acyberattack by the system 401 all available information about theongoing attack and existing cybersecurity knowledge are analyzed,including through predictive simulation in near real time 402 to developboth the most accurate appraisal of current events and actionablerecommendations concerning where the attack may progress and how it maybe mitigated. The information generated in totality is often more thanany one group needs to perform their mitigation tasks. At this point,during a cyberattack, providing a single expansive and all inclusivealert, dashboard image, or report may make identification and actionupon the crucial information by each participant more difficult,therefore the cybersecurity focused arrangement may create multipletargeted information streams each concurrently designed to produce mostrapid and efficacious action throughout the enterprise during the attackand issue follow-up reports with and recommendations or information thatmay lead to long term changes afterward 403. Examples of groups that mayreceive specialized information streams include but may not be limitedto front line responders during the attack 404, incident forensicssupport both during and after the attack 405, chief information securityofficer 406 and chief risk officer 407 the information sent to thelatter two focused to appraise overall damage and to implement bothmitigating strategy and preventive changes after the attack. Front lineresponders may use the cyber-decision platform's analyzed, transformedand correlated information specifically sent to them 404 a to probe theextent of the attack, isolate such things as: the predictive attacker'sentry point onto the enterprise's network, the systems involved or thepredictive ultimate targets of the attack and may use the simulationcapabilities of the system to investigate alternate methods ofsuccessfully ending the attack and repelling the attackers in the mostefficient manner, although many other queries known to those skilled inthe art are also answerable by the invention. Simulations run may alsoinclude the predictive effects of any attack mitigating actions onnormal and critical operation of the enterprise's IT systems andcorporate users. Similarly, a chief information security officer may usethe cyber-decision platform to predictively analyze 406 a what corporateinformation has already been compromised, predictively simulate theultimate information targets of the attack that may or may not have beencompromised and the total impact of the attack what can be done now andin the near future to safeguard that information. Further, duringretrospective forensic inspection of the attack, the forensic respondermay use the cyber-decision platform 405 a to clearly and completely mapthe extent of network infrastructure through predictive simulation andlarge volume data analysis. The forensic analyst may also use theplatform's capabilities to perform a time series and infrastructuralspatial analysis of the attack's progression with methods used toinfiltrate the enterprise's subnets and servers. Again, the chief riskofficer would perform analyses of what information 407 a was stolen andpredictive simulations on what the theft means to the enterprise as timeprogresses. Additionally, the system's predictive capabilities may beemployed to assist in creation of a plan for changes of the ITinfrastructural that should be made that are optimal for remediation ofcybersecurity risk under possibly limited enterprise budgetaryconstraints in place at the company so as to maximize financial outcome.

FIG. 5 is a diagram of an exemplary architecture for a system for rapidpredictive analysis of very large data sets using an actor-drivendistributed computational graph 500, according to one aspect. Accordingto the aspect, a DCG 500 may comprise a pipeline orchestrator 501 thatmay be used to perform a variety of data transformation functions ondata within a processing pipeline, and may be used with a messagingsystem 510 that enables communication with any number of variousservices and protocols, relaying messages and translating them as neededinto protocol-specific API system calls for interoperability withexternal systems (rather than requiring a particular protocol or serviceto be integrated into a DCG 500).

Pipeline orchestrator 501 may spawn a plurality of child pipelineclusters 502 a-b, which may be used as dedicated workers forstreamlining parallel processing. In some arrangements, an entire dataprocessing pipeline may be passed to a child cluster 502 a for handling,rather than individual processing tasks, enabling each child cluster 502a-b to handle an entire data pipeline in a dedicated fashion to maintainisolated processing of different pipelines using different cluster nodes502 a-b. Pipeline orchestrator 501 may provide a software API forstarting, stopping, submitting, or saving pipelines. When a pipeline isstarted, pipeline orchestrator 501 may send the pipeline information toan available worker node 502 a-b, for example using AKKA™ clustering.For each pipeline initialized by pipeline orchestrator 501, a reportingobject with status information may be maintained. Streaming activitiesmay report the last time an event was processed, and the number ofevents processed. Batch activities may report status messages as theyoccur. Pipeline orchestrator 501 may perform batch caching using, forexample, an IGFS™ caching filesystem. This allows activities 512 a-dwithin a pipeline 502 a-b to pass data contexts to one another, with anynecessary parameter configurations.

A pipeline manager 511 a-b may be spawned for every new runningpipeline, and may be used to send activity, status, lifecycle, and eventcount information to the pipeline orchestrator 501. Within a particularpipeline, a plurality of activity actors 512 a-d may be created by apipeline manager 511 a-b to handle individual tasks, and provide outputto data services 522 a-d. Data models used in a given pipeline may bedetermined by the specific pipeline and activities, as directed by apipeline manager 511 a-b. Each pipeline manager 511 a-b controls anddirects the operation of any activity actors 512 a-d spawned by it. Apipeline process may need to coordinate streaming data between tasks.For this, a pipeline manager 511 a-b may spawn service connectors todynamically create TCP connections between activity instances 512 a-d.Data contexts may be maintained for each individual activity 512 a-d,and may be cached for provision to other activities 512 a-d as needed. Adata context defines how an activity accesses information, and anactivity 512 a-d may process data or simply forward it to a next step.Forwarding data between pipeline steps may route data through astreaming context or batch context.

A client service cluster 530 may operate a plurality of service actors521 a-d to serve the requests of activity actors 512 a-d, ideallymaintaining enough service actors 521 a-d to support each activity perthe service type. These may also be arranged within service clusters 520a-d, in a manner similar to the logical organization of activity actors512 a-d within clusters 502 a-b in a data pipeline. A logging service530 may be used to log and sample DCG requests and messages duringoperation while notification service 540 may be used to receive alertsand other notifications during operation (for example to alert onerrors, which may then be diagnosed by reviewing records from loggingservice 530), and by being connected externally to messaging system 510,logging and notification services can be added, removed, or modifiedduring operation without impacting DCG 500. A plurality of DCG protocols550 a-b may be used to provide structured messaging between a DCG 500and messaging system 510, or to enable messaging system 510 todistribute DCG messages across service clusters 520 a-d as shown. Aservice protocol 560 may be used to define service interactions so thata DCG 500 may be modified without impacting service implementations. Inthis manner it can be appreciated that the overall structure of a systemusing an actor-driven DCG 500 operates in a modular fashion, enablingmodification and substitution of various components without impactingother operations or requiring additional reconfiguration.

FIG. 6 is a diagram of an exemplary architecture for a system for rapidpredictive analysis of very large data sets using an actor-drivendistributed computational graph 500, according to one aspect. Accordingto the aspect, a variant messaging arrangement may utilize messagingsystem 510 as a messaging broker using a streaming protocol 610,transmitting and receiving messages immediately using messaging system510 as a message broker to bridge communication between service actors521 a-b as needed. Alternately, individual services 522 a-b maycommunicate directly in a batch context 620, using a data contextservice 630 as a broker to batch-process and relay messages betweenservices 522 a-b.

FIG. 7 is a diagram of an exemplary architecture for a system for rapidpredictive analysis of very large data sets using an actor-drivendistributed computational graph 500, according to one aspect. Accordingto the aspect, a variant messaging arrangement may utilize a serviceconnector 710 as a central message broker between a plurality of serviceactors 521 a-b, bridging messages in a streaming context 610 while adata context service 630 continues to provide direct peer-to-peermessaging between individual services 522 a-b in a batch context 620.

It should be appreciated that various combinations and arrangements ofthe system variants described above (referring to FIGS. 1-7) may bepossible, for example using one particular messaging arrangement for onedata pipeline directed by a pipeline manager 511 a-b, while anotherpipeline may utilize a different messaging arrangement (or may notutilize messaging at all). In this manner, a single DCG 500 and pipelineorchestrator 501 may operate individual pipelines in the manner that ismost suited to their particular needs, with dynamic arrangements beingmade possible through design modularity as described above in FIG. 5.

Detailed Description of Exemplary Aspects

FIG. 8 is a flow diagram of an exemplary method 800 for cybersecuritybehavioral analytics, according to one aspect. According to the aspect,behavior analytics may utilize passive information feeds from aplurality of existing endpoints (for example, including but not limitedto user activity on a network, network performance, or device behavior)to generate security solutions. In an initial step 801, a web crawler115 may passively collect activity information, which may then beprocessed 802 using a DCG 155 to analyze behavior patterns. Based onthis initial analysis, anomalous behavior may be recognized 803 (forexample, based on a threshold of variance from an established pattern ortrend) such as high-risk users or malicious software operators such asbots. These anomalous behaviors may then be used 804 to analyzepotential angles of attack and then produce 805 security suggestionsbased on this second-level analysis and predictions generated by anaction outcome simulation module 125 to determine the likely effects ofthe change. The suggested behaviors may then be automaticallyimplemented 806 as needed. Passive monitoring 801 then continues,collecting information after new security solutions are implemented 806,enabling machine learning to improve operation over time as therelationship between security changes and observed behaviors and threatsare observed and analyzed.

This method 800 for behavioral analytics enables proactive andhigh-speed reactive defense capabilities against a variety ofcyberattack threats, including anomalous human behaviors as well asnonhuman “bad actors” such as automated software bots that may probefor, and then exploit, existing vulnerabilities. Using automatedbehavioral learning in this manner provides a much more responsivesolution than manual intervention, enabling rapid response to threats tomitigate any potential impact. Utilizing machine learning behaviorfurther enhances this approach, providing additional proactive behaviorthat is not possible in simple automated approaches that merely react tothreats as they occur.

FIG. 9 is a flow diagram of an exemplary method 900 for measuring theeffects of cybersecurity attacks, according to one aspect. According tothe aspect, impact assessment of an attack may be measured using a DCG155 to analyze a user account and identify its access capabilities 901(for example, what files, directories, devices or domains an account mayhave access to). This may then be used to generate 902 an impactassessment score for the account, representing the potential risk shouldthat account be compromised. In the event of an incident, the impactassessment score for any compromised accounts may be used to produce a“blast radius” calculation 903, identifying exactly what resources areat risk as a result of the intrusion and where security personnel shouldfocus their attention. To provide proactive security recommendationsthrough a simulation module 125, simulated intrusions may be run 904 toidentify potential blast radius calculations for a variety of attacksand to determine 905 high risk accounts or resources so that securitymay be improved in those key areas rather than focusing on reactivesolutions.

FIG. 10 is a flow diagram of an exemplary method 1000 for continuouscybersecurity monitoring and exploration, according to one aspect.According to the aspect, a state observation service 140 may receivedata from a variety of connected systems 1001 such as (for example,including but not limited to) servers, domains, databases, or userdirectories. This information may be received continuously, passivelycollecting events and monitoring activity over time while feeding 1002collected information into a graphing service 145 for use in producingtime-series graphs 1003 of states and changes over time. This collatedtime-series data may then be used to produce a visualization 1004 ofchanges over time, quantifying collected data into a meaningful andunderstandable format. As new events are recorded, such as changing userroles or permissions, modifying servers or data structures, or otherchanges within a security infrastructure, these events are automaticallyincorporated into the time-series data and visualizations are updatedaccordingly, providing live monitoring of a wealth of information in away that highlights meaningful data without losing detail due to thequantity of data points under examination.

FIG. 11 is a flow diagram of an exemplary method 50966 for mapping acyber-physical system graph (CPG), according to one aspect. According tothe aspect, a cyber-physical system graph may comprise a visualizationof hierarchies and relationships between devices and resources in asecurity infrastructure, contextualizing security information withphysical device relationships that are easily understandable forsecurity personnel and users. In an initial step 1101, behavioranalytics information (as described previously, referring to FIG. 8) maybe received at a graphing service 145 for inclusion in a CPG. In a nextstep 1102, impact assessment scores (as described previously, referringto FIG. 9) may be received and incorporated in the CPG information,adding risk assessment context to the behavior information. In a nextstep 1103, time-series information (as described previously, referringto FIG. 10) may be received and incorporated, updating CPG informationas changes occur and events are logged. This information may then beused to produce 1104 a graph visualization of users, servers, devices,and other resources correlating physical relationships (such as a user'spersonal computer or smartphone, or physical connections betweenservers) with logical relationships (such as access privileges ordatabase connections), to produce a meaningful and contextualizedvisualization of a security infrastructure that reflects the currentstate of the internal relationships present in the infrastructure.

FIG. 12 is a flow diagram of an exemplary method 1200 for continuousnetwork resilience scoring, according to one aspect. According to theaspect, a baseline score can be used to measure an overall level of riskfor a network infrastructure, and may be compiled by first collecting1201 information on publicly-disclosed vulnerabilities, such as (forexample) using the Internet or common vulnerabilities and exploits (CVE)process. This information may then 1202 be incorporated into a CPG asdescribed previously in FIG. 11, and the combined data of the CPG andthe known vulnerabilities may then be analyzed 1203 to identify therelationships between known vulnerabilities and risks exposed bycomponents of the infrastructure. This produces a combined CPG 1204 thatincorporates both the internal risk level of network resources, useraccounts, and devices as well as the actual risk level based on theanalysis of known vulnerabilities and security risks.

FIG. 13 is a flow diagram of an exemplary method 1300 for cybersecurityprivilege oversight, according to one aspect. According to the aspect,time-series data (as described above, referring to FIG. 10) may becollected 1301 for user accounts, credentials, directories, and otheruser-based privilege and access information. This data may then 1302 beanalyzed to identify changes over time that may affect security, such asmodifying user access privileges or adding new users. The results ofanalysis may be checked 1303 against a CPG (as described previously inFIG. 11), to compare and correlate user directory changes with theactual infrastructure state. This comparison may be used to performaccurate and context-enhanced user directory audits 1304 that identifynot only current user credentials and other user-specific information,but changes to this information over time and how the user informationrelates to the actual infrastructure (for example, credentials thatgrant access to devices and may therefore implicitly grant additionalaccess due to device relationships that were not immediately apparentfrom the user directory alone).

FIG. 14 is a flow diagram of an exemplary method 1400 for cybersecurityrisk management, according to one aspect. According to the aspect,multiple methods described previously may be combined to provide liveassessment of attacks as they occur, by first receiving 1401 time-seriesdata for an infrastructure (as described previously, in FIG. 10) toprovide live monitoring of network events. This data is then enhanced1402 with a CPG (as described above in FIG. 11) to correlate events withactual infrastructure elements, such as servers or accounts. When anevent (for example, an attempted attack against a vulnerable system orresource) occurs 1403, the event is logged in the time-series data 1404,and compared against the CPG 1405 to determine the impact. This isenhanced with the inclusion of impact assessment information 1406 forany affected resources, and the attack is then checked against abaseline score 1407 to determine the full extent of the impact of theattack and any necessary modifications to the infrastructure orpolicies.

FIG. 15 is a flow diagram of an exemplary method 1500 for mitigatingcompromised credential threats, according to one aspect. According tothe aspect, impact assessment scores (as described previously, referringto FIG. 9) may be collected 1501 for user accounts in a directory, sothat the potential impact of any given credential attack is known inadvance of an actual attack event. This information may be combined witha CPG 1502 as described previously in FIG. 11, to contextualize impactassessment scores within the infrastructure (for example, so that it maybe predicted what systems or resources might be at risk for any givencredential attack). A simulated attack may then be performed 1503 to usemachine learning to improve security without waiting for actual attacksto trigger a reactive response. A blast radius assessment (as describedabove in FIG. 9) may be used in response 1504 to determine the effectsof the simulated attack and identify points of weakness, and produce arecommendation report 1505 for improving and hardening theinfrastructure against future attacks.

FIG. 16 is a flow diagram of an exemplary method 1600 for dynamicnetwork and rogue device discovery, according to one aspect. Accordingto the aspect, an advanced cyber decision platform may continuouslymonitor a network in real-time 1601, detecting any changes as theyoccur. When a new connection is detected 1602, a CPG may be updated 1603with the new connection information, which may then be compared againstthe network's resiliency score 1604 to examine for potential risk. Theblast radius metric for any other devices involved in the connection mayalso be checked 1605, to examine the context of the connection for riskpotential (for example, an unknown connection to an internal data serverwith sensitive information may be considered a much higher risk than anunknown connection to an externally-facing web server). If theconnection is a risk, an alert may be sent to an administrator 1606 withthe contextual information for the connection to provide a concisenotification of relevant details for quick handling.

FIG. 17 is a flow diagram of an exemplary method 1700 for Kerberos“golden ticket” attack detection, according to one aspect. Kerberos is anetwork authentication protocol employed across many enterprise networksto enable single sign-on and authentication for enterprise services.This makes it an attractive target for attacks, which can result inpersistent, undetected access to services within a network in what isknown as a “golden ticket” attack. To detect this form of attack,behavioral analytics may be employed to detect forged authenticationtickets resulting from an attack. According to the aspect, an advancedcyber decision platform may continuously monitor a network 1701,informing a CPG in real-time of all traffic associated with people,places, devices, or services 1702. Machine learning algorithms detectbehavioral anomalies as they occur in real-time 1703, notifyingadministrators with an assessment of the anomalous event 1704 as well asa blast radius score for the particular event and a network resiliencyscore to advise of the overall health of the network. By automaticallydetecting unusual behavior and informing an administrator of the anomalyalong with contextual information for the event and network, acompromised ticket is immediately detected when a new authenticationconnection is made.

FIG. 18 is a flow diagram of an exemplary method 1800 for risk-basedvulnerability and patch management, according to one aspect. Accordingto the aspect, an advanced cyber decision platform may monitor allinformation about a network 1801, including (but not limited to) devicetelemetry data, log files, connections and network events, deployedsoftware versions, or contextual user activity information. Thisinformation is incorporated into a CPG 1802 to maintain an up-to-datemodel of the network in real-time. When a new vulnerability isdiscovered, a blast radius score may be assessed 1803 and the network'sresiliency score may be updated 1804 as needed. A security alert maythen be produced 1805 to notify an administrator of the vulnerabilityand its impact, and a proposed patch may be presented 1806 along withthe predicted effects of the patch on the vulnerability's blast radiusand the overall network resiliency score. This determines both the totalimpact risk of any particular vulnerability, as well as the overalleffect of each vulnerability on the network as a whole. This continuousnetwork assessment may be used to collect information about newvulnerabilities and exploits to provide proactive solutions with clearresult predictions, before attacks occur.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented onhardware or a combination of software and hardware. For example, theymay be implemented in an operating system kernel, in a separate userprocess, in a library package bound into network applications, on aspecially constructed machine, on an application-specific integratedcircuit (ASIC), or on a network interface card.

Software/hardware hybrid implementations of at least some of the aspectsdisclosed herein may be implemented on a programmable network-residentmachine (which should be understood to include intermittently connectednetwork-aware machines) selectively activated or reconfigured by acomputer program stored in memory. Such network devices may havemultiple network interfaces that may be configured or designed toutilize different types of network communication protocols. A generalarchitecture for some of these machines may be described herein in orderto illustrate one or more exemplary means by which a given unit offunctionality may be implemented. According to specific aspects, atleast some of the features or functionalities of the various aspectsdisclosed herein may be implemented on one or more general-purposecomputers associated with one or more networks, such as for example anend-user computer system, a client computer, a network server or otherserver system, a mobile computing device (e.g., tablet computing device,mobile phone, smartphone, laptop, or other appropriate computingdevice), a consumer electronic device, a music player, or any othersuitable electronic device, router, switch, or other suitable device, orany combination thereof. In at least some aspects, at least some of thefeatures or functionalities of the various aspects disclosed herein maybe implemented in one or more virtualized computing environments (e.g.,network computing clouds, virtual machines hosted on one or morephysical computing machines, or other appropriate virtual environments).

Referring now to FIG. 19, there is shown a block diagram depicting anexemplary computing device 10 suitable for implementing at least aportion of the features or functionalities disclosed herein. Computingdevice 10 may be, for example, any one of the computing machines listedin the previous paragraph, or indeed any other electronic device capableof executing software- or hardware-based instructions according to oneor more programs stored in memory. Computing device 10 may be configuredto communicate with a plurality of other computing devices, such asclients or servers, over communications networks such as a wide areanetwork a metropolitan area network, a local area network, a wirelessnetwork, the Internet, or any other network, using known protocols forsuch communication, whether wireless or wired.

In one aspect, computing device 10 includes one or more centralprocessing units (CPU) 12, one or more interfaces 15, and one or morebusses 14 (such as a peripheral component interconnect (PCI) bus). Whenacting under the control of appropriate software or firmware, CPU 12 maybe responsible for implementing specific functions associated with thefunctions of a specifically configured computing device or machine. Forexample, in at least one aspect, a computing device 10 may be configuredor designed to function as a server system utilizing CPU 12, localmemory 11 and/or remote memory 16, and interface(s) 15. In at least oneaspect, CPU 12 may be caused to perform one or more of the differenttypes of functions and/or operations under the control of softwaremodules or components, which for example, may include an operatingsystem and any appropriate applications software, drivers, and the like.

CPU 12 may include one or more processors 13 such as, for example, aprocessor from one of the Intel, ARM, Qualcomm, and AMD families ofmicroprocessors. In some aspects, processors 13 may include speciallydesigned hardware such as application-specific integrated circuits(ASICs), electrically erasable programmable read-only memories(EEPROMs), field-programmable gate arrays (FPGAs), and so forth, forcontrolling operations of computing device 10. In a particular aspect, alocal memory 11 (such as non-volatile random access memory (RAM) and/orread-only memory (ROM), including for example one or more levels ofcached memory) may also form part of CPU 12. However, there are manydifferent ways in which memory may be coupled to system 10. Memory 11may be used for a variety of purposes such as, for example, cachingand/or storing data, programming instructions, and the like. It shouldbe further appreciated that CPU 12 may be one of a variety ofsystem-on-a-chip (SOC) type hardware that may include additionalhardware such as memory or graphics processing chips, such as a QUALCOMMSNAPDRAGON™ or SAMSUNG EXYNOS™ CPU as are becoming increasingly commonin the art, such as for use in mobile devices or integrated devices.

As used herein, the term “processor” is not limited merely to thoseintegrated circuits referred to in the art as a processor, a mobileprocessor, or a microprocessor, but broadly refers to a microcontroller,a microcomputer, a programmable logic controller, anapplication-specific integrated circuit, and any other programmablecircuit.

In one aspect, interfaces 15 are provided as network interface cards(NICs). Generally, NICs control the sending and receiving of datapackets over a computer network; other types of interfaces 15 may forexample support other peripherals used with computing device 10. Amongthe interfaces that may be provided are Ethernet interfaces, frame relayinterfaces, cable interfaces, DSL interfaces, token ring interfaces,graphics interfaces, and the like. In addition, various types ofinterfaces may be provided such as, for example, universal serial bus(USB), Serial, Ethernet, FIREWIRE™, THUNDERBOLT™, PCI, parallel, radiofrequency (RF), BLUETOOTH™, near-field communications (e.g., usingnear-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fastEthernet interfaces, Gigabit Ethernet interfaces, Serial ATA (SATA) orexternal SATA (ESATA) interfaces, high-definition multimedia interface(HDMI), digital visual interface (DVI), analog or digital audiointerfaces, asynchronous transfer mode (ATM) interfaces, high-speedserial interface (HSSI) interfaces, Point of Sale (POS) interfaces,fiber data distributed interfaces (FDDIs), and the like. Generally, suchinterfaces 15 may include physical ports appropriate for communicationwith appropriate media. In some cases, they may also include anindependent processor (such as a dedicated audio or video processor, asis common in the art for high-fidelity AN hardware interfaces) and, insome instances, volatile and/or non-volatile memory (e.g., RAM).

Although the system shown in FIG. 19 illustrates one specificarchitecture for a computing device 10 for implementing one or more ofthe aspects described herein, it is by no means the only devicearchitecture on which at least a portion of the features and techniquesdescribed herein may be implemented. For example, architectures havingone or any number of processors 13 may be used, and such processors 13may be present in a single device or distributed among any number ofdevices. In one aspect, a single processor 13 handles communications aswell as routing computations, while in other aspects a separatededicated communications processor may be provided. In various aspects,different types of features or functionalities may be implemented in asystem according to the aspect that includes a client device (such as atablet device or smartphone running client software) and server systems(such as a server system described in more detail below).

Regardless of network device configuration, the system of an aspect mayemploy one or more memories or memory modules (such as, for example,remote memory block 16 and local memory 11) configured to store data,program instructions for the general-purpose network operations, orother information relating to the functionality of the aspects describedherein (or any combinations of the above). Program instructions maycontrol execution of or comprise an operating system and/or one or moreapplications, for example. Memory 16 or memories 11, 16 may also beconfigured to store data structures, configuration data, encryptiondata, historical system operations information, or any other specific orgeneric non-program information described herein.

Because such information and program instructions may be employed toimplement one or more systems or methods described herein, at least somenetwork device aspects may include nontransitory machine-readablestorage media, which, for example, may be configured or designed tostore program instructions, state information, and the like forperforming various operations described herein. Examples of suchnontransitory machine-readable storage media include, but are notlimited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks, and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM), flash memory (as is common in mobile devices andintegrated systems), solid state drives (SSD) and “hybrid SSD” storagedrives that may combine physical components of solid state and hard diskdrives in a single hardware device (as are becoming increasingly commonin the art with regard to personal computers), memristor memory, randomaccess memory (RAM), and the like. It should be appreciated that suchstorage means may be integral and non-removable (such as RAM hardwaremodules that may be soldered onto a motherboard or otherwise integratedinto an electronic device), or they may be removable such as swappableflash memory modules (such as “thumb drives” or other removable mediadesigned for rapidly exchanging physical storage devices),“hot-swappable” hard disk drives or solid state drives, removableoptical storage discs, or other such removable media, and that suchintegral and removable storage media may be utilized interchangeably.Examples of program instructions include both object code, such as maybe produced by a compiler, machine code, such as may be produced by anassembler or a linker, byte code, such as may be generated by forexample a JAVA™ compiler and may be executed using a Java virtualmachine or equivalent, or files containing higher level code that may beexecuted by the computer using an interpreter (for example, scriptswritten in Python, Perl, Ruby, Groovy, or any other scripting language).

In some aspects, systems may be implemented on a standalone computingsystem. Referring now to FIG. 20, there is shown a block diagramdepicting a typical exemplary architecture of one or more aspects orcomponents thereof on a standalone computing system. Computing device 20includes processors 21 that may run software that carry out one or morefunctions or applications of aspects, such as for example a clientapplication 24. Processors 21 may carry out computing instructions undercontrol of an operating system 22 such as, for example, a version ofMICROSOFT WINDOWS™ operating system, APPLE macOS™ or iOS™ operatingsystems, some variety of the Linux operating system, ANDROID™ operatingsystem, or the like. In many cases, one or more shared services 23 maybe operable in system 20, and may be useful for providing commonservices to client applications 24. Services 23 may for example beWINDOWS™ services, user-space common services in a Linux environment, orany other type of common service architecture used with operating system21. Input devices 28 may be of any type suitable for receiving userinput, including for example a keyboard, touchscreen, microphone (forexample, for voice input), mouse, touchpad, trackball, or anycombination thereof. Output devices 27 may be of any type suitable forproviding output to one or more users, whether remote or local to system20, and may include for example one or more screens for visual output,speakers, printers, or any combination thereof. Memory 25 may berandom-access memory having any structure and architecture known in theart, for use by processors 21, for example to run software. Storagedevices 26 may be any magnetic, optical, mechanical, memristor, orelectrical storage device for storage of data in digital form (such asthose described above, referring to FIG. 19). Examples of storagedevices 26 include flash memory, magnetic hard drive, CD-ROM, and/or thelike.

In some aspects, systems may be implemented on a distributed computingnetwork, such as one having any number of clients and/or servers.Referring now to FIG. 21, there is shown a block diagram depicting anexemplary architecture 30 for implementing at least a portion of asystem according to one aspect on a distributed computing network.According to the aspect, any number of clients 33 may be provided. Eachclient 33 may run software for implementing client-side portions of asystem; clients may comprise a system 20 such as that illustrated inFIG. 20. In addition, any number of servers 32 may be provided forhandling requests received from one or more clients 33. Clients 33 andservers 32 may communicate with one another via one or more electronicnetworks 31, which may be in various aspects any of the Internet, a widearea network, a mobile telephony network (such as CDMA or GSM cellularnetworks), a wireless network (such as WiFi, WiMAX, LTE, and so forth),or a local area network (or indeed any network topology known in theart; the aspect does not prefer any one network topology over anyother). Networks 31 may be implemented using any known networkprotocols, including for example wired and/or wireless protocols.

In addition, in some aspects, servers 32 may call external services 37when needed to obtain additional information, or to refer to additionaldata concerning a particular call. Communications with external services37 may take place, for example, via one or more networks 31. In variousaspects, external services 37 may comprise web-enabled services orfunctionality related to or installed on the hardware device itself. Forexample, in one aspect where client applications 24 are implemented on asmartphone or other electronic device, client applications 24 may obtaininformation stored in a server system 32 in the cloud or on an externalservice 37 deployed on one or more of a particular enterprise's oruser's premises.

In some aspects, clients 33 or servers 32 (or both) may make use of oneor more specialized services or appliances that may be deployed locallyor remotely across one or more networks 31. For example, one or moredatabases 34 may be used or referred to by one or more aspects. Itshould be understood by one having ordinary skill in the art thatdatabases 34 may be arranged in a wide variety of architectures andusing a wide variety of data access and manipulation means. For example,in various aspects one or more databases 34 may comprise a relationaldatabase system using a structured query language (SQL), while othersmay comprise an alternative data storage technology such as thosereferred to in the art as “NoSQL” (for example, HADOOP CASSANDRA™,GOOGLE BIGTABLE™, and so forth). In some aspects, variant databasearchitectures such as column-oriented databases, in-memory databases,clustered databases, distributed databases, or even flat file datarepositories may be used according to the aspect. It will be appreciatedby one having ordinary skill in the art that any combination of known orfuture database technologies may be used as appropriate, unless aspecific database technology or a specific arrangement of components isspecified for a particular aspect described herein. Moreover, it shouldbe appreciated that the term “database” as used herein may refer to aphysical database machine, a cluster of machines acting as a singledatabase system, or a logical database within an overall databasemanagement system. Unless a specific meaning is specified for a givenuse of the term “database”, it should be construed to mean any of thesesenses of the word, all of which are understood as a plain meaning ofthe term “database” by those having ordinary skill in the art.

Similarly, some aspects may make use of one or more security systems 36and configuration systems 35. Security and configuration management arecommon information technology (IT) and web functions, and some amount ofeach are generally associated with any IT or web systems. It should beunderstood by one having ordinary skill in the art that anyconfiguration or security subsystems known in the art now or in thefuture may be used in conjunction with aspects without limitation,unless a specific security 36 or configuration system 35 or approach isspecifically required by the description of any specific aspect.

FIG. 22 shows an exemplary overview of a computer system 40 as may beused in any of the various locations throughout the system. It isexemplary of any computer that may execute code to process data. Variousmodifications and changes may be made to computer system 40 withoutdeparting from the broader scope of the system and method disclosedherein. Central processor unit (CPU) 41 is connected to bus 42, to whichbus is also connected memory 43, nonvolatile memory 44, display 47,input/output (I/O) unit 48, and network interface card (NIC) 53. I/Ounit 48 may, typically, be connected to keyboard 49, pointing device 50,hard disk 52, and real-time clock 51. NIC 53 connects to network 54,which may be the Internet or a local network, which local network may ormay not have connections to the Internet. Also shown as part of system40 is power supply unit 45 connected, in this example, to a mainalternating current (AC) supply 46. Not shown are batteries that couldbe present, and many other devices and modifications that are well knownbut are not applicable to the specific novel functions of the currentsystem and method disclosed herein. It should be appreciated that someor all components illustrated may be combined, such as in variousintegrated applications, for example Qualcomm or Samsungsystem-on-a-chip (SOC) devices, or whenever it may be appropriate tocombine multiple capabilities or functions into a single hardware device(for instance, in mobile devices such as smartphones, video gameconsoles, in-vehicle computer systems such as navigation or multimediasystems in automobiles, or other integrated hardware devices).

In various aspects, functionality for implementing systems or methods ofvarious aspects may be distributed among any number of client and/orserver components. For example, various software modules may beimplemented for performing various functions in connection with thesystem of any particular aspect, and such modules may be variouslyimplemented to run on server and/or client components.

The skilled person will be aware of a range of possible modifications ofthe various aspects described above. Accordingly, the present inventionis defined by the claims and their equivalents.

What is claimed is:
 1. An advanced cyber decision platform for externalnetwork reconnaissance and cybersecurity rating, the platformcomprising: a computing device comprising a memory and a processor; atime-series data module comprising a first plurality of programminginstructions stored in the memory and operating on the processor,wherein the first plurality of programming instructions, when operatingon the processor, cause the computing device to: receive cybersecurityreconnaissance data about a company from a web crawler; producetime-series data based on at least a portion of the cybersecurityreconnaissance data; a directed computational graph module comprising asecond plurality of programming instructions stored in the memory andoperating on the processor, wherein the second plurality of programminginstructions, when operating on the processor, cause the computingdevice to: retrieve at least a portion of the time-series data; receivea directed computational graph for performing a plurality oftransformation operations on at least a portion of the time-series data,wherein the directed computational graph comprises nodes and edges, thenodes representing the transformation operations and the edgesrepresenting message outputs between the nodes; and produce a weightedcybersecurity score based on at least a portion of the transformationoperations, wherein the weighted cybersecurity score represents a riskof external cyber-security threats to the company based on the collectedcybersecurity reconnaissance data; and a web crawler comprising a thirdplurality of programming instructions stored in the memory and operatingon the processor, wherein the third plurality of programminginstructions, when operating on the processor, cause the computingdevice to: collect cybersecurity reconnaissance data from websites andweb applications relevant to the company, the collection ofcybersecurity reconnaissance data comprising: identifying portions ofeach website or web application that web crawlers are requested toignore; fingerprinting each website or web application to identifycharacteristic patterns or markers that may be used to identify host orapplication details; and checking each web application and website'sadministrative pages to determine if any administrative portals arevulnerable to cyber-attacks; and provide the collected leak informationand the reconnaissance data to the time-series data module.
 2. A methodfor external network reconnaissance and cybersecurity rating, comprisingthe steps of: using a web crawler to: collect cybersecurityreconnaissance data for websites and web applications relevant to acompany, the collection of cybersecurity reconnaissance data comprising:identifying portions each website or web application that web crawlersare requested to ignore; fingerprinting each website or web applicationto identify characteristic patterns or markers that may be used toidentify host or application details; and checking each web applicationand each website's administrative pages to determine if anyadministrative portals are vulnerable to cyber-attacks; producing, usinga time-series data module, time-series data based on at least a portionof the collected cybersecurity reconnaissance data; receive a directedcomputational graph for performing a plurality of transformationoperations on at least a portion of the time-series data, wherein thedirected computational graph comprises nodes and edges, the nodesrepresenting the transformation operations and the edges representingmessage outputs between the nodes; and produce a weighted score based onat least a portion of the transformation operations, wherein theweighted score represents a risk of external cyber-security threats tothe company based on the collected cybersecurity reconnaissance data.